Comparison of classical test theory and item response theory in terms of item parameters

There are two currently popular statistical frameworks for addressing measurement problems namely Classical Test Theory (CTT) and Item Response Theory (IRT). This study is aimed empirically to examine the similarities and differences in the parameters estimated using these two frameworks. For this purpose, firstly the model assumptions and the goodness of fit analyses were checked for IRT. The results suggested that the most appropriate model which fit the data was achieved by three parameter logistic IRT model. Then item difficulty and item discrimination parameters were calculated in terms of both IRT and CTT. Lastly parameters which were obtained by CTT and IRT analyses were compared. In the study, the answers given to the 25-item Turkish test by randomly chosen 1250 students from the group of 5989 students who had taken the high schools entrance exam (HSEE) in 2003 were used. In consequence, it was found that the highest correlations were available between CTT and 1parameter IRT model (0.99) in terms of item difficulty parameters, and between CTT and 2-parameter IRT model (0.96) in terms of item discrimination parameters. Besides, although the 3-parameter model was identified as the most congruous one in terms of model-data fit, the lowest level of correlation was found between the 3-parameter model and CTT. In the light of these findings, it may be said that there is not much difference between using 1 or 2-parameter IRT model and CTT. However, in cases where the probability of guessing is high, there is a significant difference between 3-parameter model and CTT. © 2014 European Journal of Research on Education by IASSR.